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ANALYSIS  OF  MULTI-CRITERIA  FIRE  DETECTION  SYSTEM  DATA  AND  EARLY 
WARNING  FIRE  DETECTION  PROTOTYPE  SELECTION 


1.  INTRODUCTION 

A  series  of  tests  were  conducted  to  evaluate  sensors  for  an  early  warning  fire  detection 
system  under  development.  The  tests  were  conducted  from  August  30-September  3,  1999 
onboard  the  ex-USS  SHADWELL,  the  Naval  Research  laboratory’s  full  scale  fire  research 
facility  in  Mobile,  Alabama  (reference  (a)).  The  tests  have  been  used  to  evaluate  and  improve 
the  multivariate  data  analysis  methods  and  candidate  sensor  suites  described  in  references  (b- 
d).  The  objective  of  the  program  is  to  develop  an  improved  early  warning  fire  detection 
system  that  will  provide  early  fire  detection  with  a  low  false  alarm  rate. 

2.  BACKGROUND 

The  system  under  development  combines  a  multi-criteria  (sensor  array)  approach  with 
sophisticated  data  analysis  methods.  Together  an  array  of  sensors  and  a  multivariate 
classification  algorithm  produce  early  fire  detection  with  a  low  false  alarm  rate.  Several 
sensors  measuring  different  parameters  of  the  environment  produce  a  pattern  or  response 
fingerprint  for  an  event.  Multivariate  data  analysis  methods  are  trained  to  recognize  the 
pattern  of  an  important  event  such  as  a  fire.  Multivariate  methods  are  trained  using  data  in  a 
training  set  and  the  training  set  consists  of  sensor  responses  to  events  and  nonevents  under 
various  conditions.  The  data  sets  used  for  sensor  array  evaluation  require  that  the  sensors  be  in 
close  proximity  so  that  it  can  be  assumed  that  the  sensors  are  observing  the  same  test 
atmosphere. 

Multivariate  classification  methods  rely  on  the  comparison  of  events  with  nonevents. 
Variations  in  the  response  of  sensors  can  be  used  to  train  an  algorithm  to  recognize  events 
when  they  occur.  A  key  to  the  success  of  these  methods  is  the  appropriate  design  of  sensor 
arrays  and  the  training  sets  used  to  develop  the  algorithm.  Although,  the  event  is  most 
important,  it  is  critical  that  the  algorithms  recognize  nonevents  as  well. 

Standard  test  procedures  included  a  baseline  response  to  establish  initial  sensor  conditions, 
exposure  test,  and  recovery  back  to  baseline.  For  example,  a  typical  test  collected  10  minutes 
of  ambient  air,  followed  by  an  exposure  to  a  fire  for  20  minutes,  then  re-exposure  to  ambient 
conditions  for  10  minutes.  Chemical  sensors  are  subject  to  noise  and  other  fluctuations  with 
time.  Therefore,  typical  sensor  performance  or  typical  baseline  responses  over  the  test  period 
were  determined  by  exposing  the  sensors  to  ambient  air  for  the  entire  test  period.  Baseline 
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tests  conducted  at  different  times  of  the  day  can  be  used  to  determine  environmental  effects 
and  how  they  influence  the  sensors. 

Data  were  collected  by  the  multi-criteria  fire  detection  system  on  the  ex-USS 
SHADWELL.  These  data  were  used  to  validate  the  performance  of  the  system  with  respect  to 
the  laboratory  database  (ref  (b))  and  to  optimize  the  sensor  array  selection  for  prediction.  This 
report  describes: 

(1)  The  evaluation  of  the  candidate  suites  and  the  probabilistic  neural  networks  for 
early  and  reliable  detection  of  several  types  of  fires  (i.e.,  validation  database); 

(2)  The  database  that  was  used  to  improve  the  classifier  for  shipboard  use  (i.e.,  use  the 
database  as  a  secondary  training  set  if  the  validation  is  not  satisfactory);  and 

(3)  The  reliability  of  the  multi-sensor  detection  system  with  respect  to  nuisance  alarms. 

The  detection  system  was  installed  in  the  forward  area  of  the  ship  on  the  second  deck  FR 
15-22  as  shown  in  Figure  1  along  with  selected  SHADWELL  sensors  including  thermocouples 
and  continuous  oxygen  (O2),  carbon  monoxide  (CO)  and  carbon  dioxide  (CO2)  gas  sampling. 
The  SHADWELL  sensors  were  used  to  verily  the  detection  system.  The  location  of  the  fire  is 
also  shown  in  Figure  1.  The  standard  test  procedure  was  used  and  included  10  minutes  of 
ambient  background  air,  followed  by  an  exposure  to  a  fire  for  a  fixed  finite  time  (20  minutes), 
and  then  re-exposure  to  ambient  conditions  (10  minutes).  Background  tests  were  conducted  by 
exposing  the  sensors  to  the  ambient  condition  for  the  entire  test  period  (40  minutes).  After 
every  2  fire  tests,  a  nuisance  source  test  was  conducted,  so  that  the  fire  and  nuisance  tests 
were  intermixed.  References  (e  and  f)  describe  the  field  test  parameters  and  the  results. 

The  sensors  were  calibrated  prior  to  installation  with  calibration  gas  mixtures  diluted  to  an 
appropriate  level  using  clean  breathing  air  (upper  limit  of  sensor  range).  The  calibration  of  the 
sensors  was  verified  daily,  while  the  sensors  were  mounted  in  place. 

3.  FIRE  SCENARIOS 

A  total  of  30  tests  were  conducted  to  evaluate  the  detection  system.  Fire  scenarios  used 
common  shipboard  combustibles,  such  as  oily  rags,  cardboard,  paper,  sheets,  and  mattresses 
as  the  fuel.  Nuisance  source  scenarios  represented  common  shipboard  activities,  such  as 
welding,  cutting  steel  with  a  torch,  toasting,  smoking,  cleaning,  personal  products,  and 
burning  popcorn.  The  majority  of  the  sources  were  located  at  2-17-0.  Table  1  summarizes 
the  fire/nuisance  source  scenarios  and  other  general  test  conditions  for  each  test.  Replicates 
were  not  tested  sequentially.  The  small  heptane  pan  fire  was  used  as  our  standard  test  and  was 
used  periodically  to  determine  the  reproducibility  and  the  stability  of  the  sensors  during  the 
test  series. 

Table  2  presents  a  list  of  the  sensors  used  in  the  test  program.  Under  the  column  labeled 
species,  the  parenthetical  term  represents  the  sensor  name  used  throughout  this  program.  The 
majority  of  the  gas  sensors  were  electrochemical  cell  technology,  except  as  noted  below. 

These  sensors  were  used  because  they  provided  a  means  to  economically  measure  many 
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species.  Past  experience  with  the  CO  sensors  indicated  that  these  sensors  are  accurate  at  low 
ppm  concentrations,  are  easy  to  operate  and  calibrate  and  are  reliable  over  repetitive  testing. 
The  general  hydrocarbon  sensor  (calibrated  with  ethylene)  was  a  solid  state  metal  oxide 
sensor.  The  COj  meter  was  designed  for  indoor  air  quality  measurements  based  on  non- 
dispersive  infrared  (NDIR)  technology.  All  of  the  gas  sensors  operated  via  gas  diffusion  to  the 
unit. 

At  the  beginning  of  each  day,  the  daily  checklist  was  completed.  Prior  to  each  test,  the 
fire  area  was  cleared  of  all  personnel  not  involved  with  testing.  All  hatches  and  doors  were 
closed.  Ventilation  to  the  space  remained  off  for  the  first  30  minutes  of  the  test.  After 
completion  of  these  tasks,  test  personnel  were  positioned  in  the  appropriate  locations.  When 
the  fuel  package  was  ready  and  the  safety  team  was  in  position,  data  collection  and  videos 
were  initiated  to  commence  the  test.  Following  approximately  10  minutes  of  background  data, 
the  source  was  initiated.  For  the  smoldering  fire  scenarios,  the  soldering  irons  were 
energized.  Event  data  were  collected  for  20  minutes.  After  the  event  was  secured,  and  during 
compartment  venting,  the  data  collection  continued  for  10  additional  minutes  to  assess  the 
recovery  of  the  sensors  following  an  event.  Once  the  Safety  Team  had  deemed  the  test  area 
safe  for  personnel  without  breathing  protection,  the  test  area  was  prepared  for  the  next  test. 
This  preparation  included  any  cleanup  of  the  test  area,  equipment  setup  for  the  next  test,  and 
verification  of  instruments. 


All  of  the  sensor  responses  were  collected  using  the  MASSCOMP  on  the  SHADWELL. 


*  SHADWELLGAS  SENSOR 


\ 

\ 


4 


Figure  1 .  Diagram  of  the  forward  area  of  the  ship  on  the  second 
deck  between  FR  1 5-22  showing  the  sensor  and  fire  locations. 


Table  1.  Summary  of  Test  Scenarios 


TEST 

FIRE  SCENARIO 

COMMENTS 

MV-00 

Background 

MV-01 

Flaming  heptane  in  15  cm  (6,0  in.)  diameter 
pan. 

Approximately  100  ml  (3.5  fl.  oz.)  heptane  in 
pan,  Standard  Test 

MV-02 

Flaming  heptane  in  15  cm  (6.0  in.)  diameter 
pan. 

Repeat  MV-01,  Standard  Test 

MV-03 

Background 

Repeat  MV-00 

MV-04 

Flaming  heptane  in  15  cm  (6.0  in.)  diameter 
pan. 

Repeat  MV-Ol,  Standard  Test 

MV-05 

Flaming  oily  rags  in  small  trash  can 

0.1  m^  (1  ft^)  rags  saturated  with  118  ml  (4  oz.) 
10W30  motor  oil,  ignited  with  a  butane  lighter 

MV-06 

Burning  Toast 

4  slices  of  bread  in  a  toaster  locked  on 

MV-07 

Flaming  paper  and  cardboard  in  small  trash  can 

5  sheets  of  newspaper  and  0.3  m^  (3  ft^)  of  cardboard, 
ignited  with  a  butane  lighter 

MV-08 

Smoldering  oily  rags  in  small  trash  can 

0.1  m^  (1  ft^)  rags  saturated  with  118  ml  (4  oz.) 
10W30  motor  oil,  ignited  with  a  butane  lighter 

MV-09 

Burning  Popcorn 

1  bag  of  popcorn  in  microwave  for  12  minutes 

MV-10 

Flaming  oily  rags  in  small  trash  can 

Repeat  of  MV-05 

MV- 11 

Flaming  paper  and  cardboard  in  small  trash  can 

Repeat  of  MV-07 

MV-12 

Welding 

MV- 13 

Background 

Repeat  MV-00 

Smoldering  paper  in  small  trash  can 

5  sheets  of  newspaper  and  0.3  m^  (3  ft^)  of  cardboard, 
ignited  with  a  butane  lighter 

MV-15 

Smoldering  cotton  sheet,  pillow,  wool  blanket, 
and  mattress 

Fuel  package  heated  with  a  300  W  heating  coil 
energized  to  54  V.  Coil  on  top  center  under  its  own 
weight. 

MV-16 

Cutting  Steel  with  an  Acetylene  Torch 

MV-17 

Flaming  fuel  oil  in  0.3  m  x  0.3  m  (1  ft  x  1  ft) 
square  pan 

1.1  liters  (0.3  gal)  F-76  with  ethyl  alcohol  accelerant 

MV-18 

Flaming  0.3  m  x  0.3  m  x  0.2  m 
(1  ft  X  1  ft  X  10  in.)  wood  crib 

Crib  constructed  with  4  rows  of  4  -  5 1  mm  x  5 1  mm  x 
0,2  m  sticks.  Crib  ignited  with  a  small  heptane  pan 
fire 

MV-19 

Cleaning  Supplies 

MV-20 

Smoldering  pillow  in  a  pillow  case,  15.2  cm  x 
15.2  cm  (6  in.  x  6  in.) 

Fuel  package  heated  with  a  300  W  heating  coil 
energized  to  54  V.  Coil  on  top  center  under  its  own 
weight. 

MV- 

2V 

Flaming  heptane  in  15  cm  (6.0  in.)  diameter 
pan. 

MV-01,  Standard  Test 

MV-22 

Cigarette  Smoke 

MV-23 

TODCO  Wall  Board  exposed  to  a  flame 

TODCO  wallboard,  10  cm  x  30  cm  (4  in,  x  1  ft.) 
exposed  to  a  flame 

MV-24 

Pipe  Insulation  exposed  to  flame 

Calcium  silicate  insulation  with  glass  cloth  lagging, 
painted  (45  cm)  exposed  to  a  flame 

MV-25 

Personal  care  products 

Rubbing  alcohol,  Ben-Gay,  shaving  cream,  and 
Tinactin 

Mvsm 

Background 

Repeat  MV-00 

n 

Smoldering  cotton  sheet,  pillow,  wool  blanket, 
and  mattress 

Repeat  MV-15 

MV-28 

Smoldering  cables  in  overhead  cable  rack 

LSDSGU-14  cable  (33  cm)  ohmically  heated  with  a 
600  A  arc  welder 

MV-29 

Background 

Repeat  MV-00 
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Table  2.  Sensors  for  Multi-criteria  Detection  Tests 


No. 

Species 

Sensor  Range 

Resolution 

Model  No. 

Manufacturer 

■ 

Oxygen  (Oj) 

0-25% 

0.1%  O2 

6C 

City  Technology 

■ 

Carbon  monoxide  (CO20000 

ppm) 

0-20000  ppm 

5  ppm 

City  Technology 

■ 

Carbon  monoxide  w/  Hj 
compensation  (C04oooppn.) 

0-4000  ppm 

1  ppm 

A3ME/F 

City  Technology 

D 

Carbon  monoxide  (CO50ppm) 

0-50  ppm 

0.5  ppm 

TB7E-1A 

City  Technology 

5 

Carbon  dioxide  (CO2) 

0-5(X)0  ppm 

Accuracy  =  greater 
of  ±5%  of  reading 
or  -f  100  ppm 

2001V 

Telairc/Englehard 

6 

Cl  to  C^  (ETHYL) 
Hydrocarbons 
(will  be  calibrated  with 
ethylene) 

0-50  ppm 

+  2.5  ppm 

SM95-S2  with 
general 
hydrocarbons 
solid  state  sensor 

International 

Sensor 

Technology 

7 

C,  to  C,  (ETHYL) 
Hydrocarbons 

0-20000  ppm 

8 

Nitric  oxide  (NO) 

0-20  ppm 

0.5  ppm 

TF3C-1A 

City  Technology 

9 

Hydrogen  chloride  (HCl) 

0-10  ppm 

0.5  ppm 

TLIB-IA 

City  Technology 

10 

Temperature  (Thermocouple 
orTC) 

-200  to  1250EC 

lEC  or  0.75% 

Type  K,  0.127 
mm  bare  bead 

TC 

Omega 

11 

Temperature  (Temp  Omega) 

-20EC  to  75EC 

+0.6EC 

accuracy 

HX93  transmitter 
(RTD) 

Omega 

12 

Relative  humidity  (RH) 

3-95% 

J:2%  RH 
accuracy 

HX93  transmitter 

Omega 

■ 

Hydrogen  Sulfide  (H2S) 

City  Technology 

■ 

Photoelectric  smoke  detector 
(PHOTO) 

0-19%  Obs/m 

4098-9701 

Simplex 

15 

Ionization  smoke  detector 
(ION) 

1.6-10% 

Obs/m 

4098-9716 

Simplex 

■ 

Residential  ionization  smoke 
detector  (RION) 

83R 

First  Alert 

17 

Optical  Density  Meter 
(ODM)  (1  m  path  length) 

Infrared  light 
emitting  diode 
and  receptor 

Meredith 

■ 

Optical  Density  Meter 
(ODM)  (1  m  path  length) 

Laser  and 
Photodiode 

Motorola 

19 

Optical  Density  Meter 
(ODM)  (1  m  path  length) 

White  Light 
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4.  DATA  ANALYSIS 


The  data  were  received  in  two  forms:  raw  and  processed.  The  raw  data  were  taken 
directly  off  the  MASSCOMP.  The  processed  data  included  engineering  units,  and  the  Optical 
Density  Meter  (ODM)  and  Simplex  smoke  detectors  (PHOTO  and  ION)  were  in  percent 
obscuration  per  meter  (%  obs/m)  as  described  in  reference  (b).  The  first  minute  of 
background  was  used  for  the  initial  intensity  in  the  %  obs/m  calculation.  The  data  from  the 
Simplex  sensors  were  collected  every  four  to  five  seconds,  while  all  the  other  sensors  provided 
data  every  second.  For  pattern  recognition,  all  of  the  data  responses  were  consolidated  into  a 
data  matrix  containing  all  the  sensor  responses.  Therefore,  only  the  responses  occurring  at  the 
same  time  as  the  Simplex  sensors  were  included  in  the  data  matrix.  The  data  sets  were 
examined  directly  to  determine  the  best  parameters  for  shipboard  detection.  In  addition,  these 
tests  were  used  as  a  prediction  set  to  validate  the  algorithms  developed  using  the  laboratory 
data  sets  described  in  references  (b-d).  For  this  application,  strong  nuisance  source  rejection 
was  given  the  highest  consideration. 

The  data  were  analyzed  on  a  PC  using  routines  written  in  MATLAB,  version  5.2 
(Mathworks,  Inc.,  Natick,  MA).  Many  of  the  routines  were  implemented  using  the 
PLS  toolbox,  version  2.0c  (Eigenvector  Technologies,  Inc.,  Manson,  WA).  All  the  matrices 
were  autoscaled  (column  mean  of  zero,  unit  variance).  The  classifier  used  in  this  study  was  a 
Probabilistic  Neural  Network  (PNN)  (reference  (g))  that  was  developed  at  the  Naval  Research 
laboratory  for  chemical  sensor  arrays.  A  graphical  user  interface  (GUI)  for  use  with 
MATLAB  entitled  MATLAB  Graphical  Interface  for  Classification  Algorithms  (MAGICAL) 
was  developed  for  easier  implementation  of  PNN  routines.  The  software  has  the  capability  to 
run  the  PNN  in  cross-validation  and  Venetian  blinds  (VB)  mode  (random  selection  of  training 
and  prediction  sets  from  a  single  data  set).  The  GUI  has  functionality  that  enables  data 
visualization,  manual  and  automated  variable  selection,  sigma  calculation  and  optimization, 
results  plotting  (PNN  probabilities)  and  experiment  /  data  storage.  The  PNN  operates  by 
defining  a  probability  density  function  (PDF)  for  each  data  class  based  on  the  training  set  data 
and  the  optimized  kernel  width  parameter.  The  PDF  defines  the  boundaries  for  each  data 
class.  For  classifying  new  events,  the  PDF  is  used  to  estimate  the  probability  that  the  new 
pattern  belongs  to  each  data  class.  The  output  probability  can  be  used  as  a  level  of  confidence 
in  the  classification  decision  and  can  serve  as  a  guide  to  reducing  the  false  alarm  rate.  Most  of 
the  studies  organized  the  data  into  three  classes  for  analysis:  fire,  nonfire  (background),  and 
nuisance  sources. 

A  new  stepwise  variable  selection  routine  specifically  designed  for  the  PNN  classifier  was 
developed  and  used  to  select  subsets  of  sensors.  The  code,  called  VARSELPR,  used  a 
straightforward  implementation  of  forward  selection  using  the  sum  of  squared  error  from  the 
PNN-cross  validation  algorithm  as  the  criteria  for  the  best  sensor  selection.  The  PNN  variable 
selection  routine  allows  the  user  to  select  the  number  of  sensors  to  be  included  in  the  set  and 
provides  a  mechanism  to  withhold  sensors  from  consideration  in  the  array. 
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Two  approaches  were  used  to  analyze  the  processed  and  raw  data  sets.  The  processed  data 
related  most  closely  to  the  analyses  in  references  (b  and  c),  while  the  raw  data  analysis 
methods  corresponded  most  closely  to  the  studies  described  in  reference  (d).  Both  methods 
are  given  here  for  completeness.  However,  the  raw  data  set  methods  are  more  relevant  to 
real-time  detection  so  they  are  used  to  determine  the  final  candidate  suites  and  to  optimize  the 
algorithms  for  real-time  monitoring. 

In  the  previous  studies  that  investigated  the  laboratory  data  sets  (references  (b-d)),  several 
sets  of  sensors  were  found  to  be  useful  for  accurate  fire  detection  and  nuisance  source 
rejection.  Similar  experiments  were  conducted  to  determine  whether  these  same  sets  of  sensors 
are  important  for  the  SHADWELL  field  data.  To  distinguish  these  experiments  from  ones 
where  the  laboratory  data  set  is  the  training  set  and  the  SHADWELL  field  data  is  the 
prediction  set,  these  experiments  will  be  termed,  intra-set  cross  validation  because  training 
and  validation  is  done  using  sensor  data  from  only  one  site.  The  term,  inter-set  prediction  will 
be  reserved  to  specify  training  on  data  from  one  site  and  prediction  of  another. 

The  last  section  of  the  report  describes  the  experiments  used  to  define  two  prototypes  for 
testing  on  the  SHADWELL  in  real  time  and  the  optimization  of  the  algorithms  for  that 
application.  In  those  experiments,  the  entire  SHADWELL  test  was  used  in  a  playback  mode. 

4.1  Processed  Data  Analysis 

Using  the  SHADWELL  data,  training  and  prediction  sets  were  constructed  at  distinct  times 
using  the  alarm  times  for  the  Simplex  Photoelectric  detector  on  Board  A  at  the  11,  1.63,  and 
0.82  %  obs/m  sensitivity  levels.  A  pattern  or  fingerprint  of  the  event  for  each  sensor  was 
generated  at  these  response  times.  The  patterns  were  based  on  the  average  sensor  readings  for 
the  last  10  data  points  and  the  rate  of  change  (slope)  from  the  last  25  values.  For  these 
experiments  the  training  sets  were  taken  from  three  points  in  time  from  each  experiment. 

Thus,  there  are  three  patterns  in  the  training  set  from  each  experiment  resulting  in  a  total  of  90 
pattern  vectors.  Background  patterns  were  taken  at  two  discrete  times:  two  minutes  after  the 
data  acquisition  was  initiated  and  a  few  seconds  prior  to  ignition.  For  real  fire  events  and 
nuisance  sources,  the  pattern  was  computed  using  data  collected  at  the  given  alarm  times.  For 
background  events  (mvOO,  mv03,  mvl3,  mv26,  and  mv29)  and  nuisance  sources  that  did  not 
cause  a  false  alarm,  the  third  pattern  was  computed  five  minutes  after  source  initiation  or  ten 
minutes  after  the  start  of  data  acquisition.  For  fire  events  that  did  not  trigger  an  alarm  at  the 
most  sensitive  setting,  the  maximum  reading  of  the  Simplex  Photoelectric  detector  on  Board  A 
was  used  to  set  the  time  at  which  the  sensor  fingerprint  was  evaluated.  In  all  other  cases,  the 
alarm  time  from  the  next  most  sensitive  level  was  used. 

The  initial  tests  used  the  SHADWELL  field  data  as  a  prediction  set  for  the  laboratory 
training  set  at  the  0.82%  obs/m  level.  The  data  set  consisted  of  90  events,  65  backgrounds, 

18  fires  and  7  nuisance  sources.  The  results  are  shown  in  Table  3a.  These  tests  demonstrate 
the  ability  to  train  an  algorithm  with  data  from  one  location  and  predict  data  from  another 
location  under  different  environmental  conditions,  an  inter-set  prediction.  There  are  some 
differences  between  the  preprocessing  of  the  laboratory  data  set  (reference  (b))  and  the 
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SHADWELL  data  set.  It  is  likely  that  these  differences  led  to  numerical  instabilities  in  the 
PNN.  The  laboratory  data  was  filtered  using  a  Savitzky-Golay  smoothing  routine;  this  was 
not  used  for  the  SHADWELL  data  because  the  way  it  was  implemented  in  the  training  set  was 
not  applicable  to  real-time  processing  that  will  occur  later  in  the  program.  Due  to  differences 
in  the  data  acquisition,  the  residential  ionization  sensor  (RION)  was  not  scaled  in  the  same 
way,  so  there  was  a  mismatch  in  the  training  and  prediction  sets.  In  addition,  the  rate  of 
change  was  not  used  with  the  preprocessed  data  due  to  the  differences  in  the  methods  used  to 
calculate  it. 

Correct  classification  results  range  from  20-86%.  The  results  are  very  low  for  some  of  the 
sensor  combinations,  particularly  ones  containing  relative  humidity  (RH),  O2  and  hydrogen 
(H2)  sensors.  It  appears  that  magnitudes  are  a  problem  for  this  application  because  the  oxygen 
and  humidity  levels  were  very  different  between  the  two  sites.  When  RH  was  in  the  sensor 
array,  most  of  the  experiments  were  classified  as  a  fire.  In  addition,  the  hydrogen  sensor 
changed  its  baseline  response  dramatically  during  the  SHADWELL  tests.  The  best  performing 
results  were  for  ION,  PHOTO,  and  ODM  sensor  combinations.  It  is  important  to  note  that 
each  of  the  sensors  was  converted  to  %  obs/m,  and  thus  have  a  form  of  background 
correction.  Therefore,  it  was  determined  that  background  correction  may  be  necessary. 

For  the  background  corrected  studies,  the  background  corrected  data  matrix  used  in 
univariate  analysis  described  in  reference  (b)  was  used.  The  data  set  consisted  of  82  fires  and 
38  nuisance  sources.  One  minute  of  background  was  subtracted  from  the  remainder  of  the 
data.  It  was  necessary  to  treat  the  data  as  a  two-class  problem;  fires  and  nuisance  sources 
because  all  of  the  limited  background  data  collected  in  those  tests  were  used  in  the  correction. 
The  laboratory  background  corrected  data  matrix  was  studied  using  the  leave-one-out  cross- 
validation  method  (PNNCV)  that  sequentially  trains  all  but  one  observation  and  predicts  the 
one  that  was  left  out,  an  intra-set  analysis  approach.  This  procedure  is  repeated  until  all  the 
observations  or  tests  have  been  predicted.  The  results  are  shown  in  Table  3b. 

Using  the  VARSELPR,  CO4000,  RH,  temperature  (TC),  ODM  and  PHOTO  were  found  to 
be  the  best  set  of  sensors.  This  set  achieved  89%  correct  classification  of  the  data  set, 
correctly  classifying  72  of  the  82  fires  and  35  of  the  38  nuisances  sources.  The  CO4000  sensor 
was  not  functioning  during  the  SHADWELL  test;  therefore,  the  CO50  sensor  was  used  in  the 
prediction.  When  we  change  to  COjg,  the  performance  of  the  training  set  is  reduced  to  83%, 
due  to  increased  nuisance  source  alarms.  The  results  are  similar  to  those  observed  in  the  early 
work,  the  three-class  approach  including  background  responses  for  the  sensors.  When  the  best 
set  (O2,  H2S,  RH,  ION,  and  PHOTO)  from  reference  (c)  was  used  in  this  two-class  approach, 
18  events  were  missed,  producing  85%  correct  classification.  The  missed  events  were  4% 
higher  than  in  the  previous  work  using  a  three-class  approach  (fire,  nonfire,  and  nuisance 
sources),  where  the  same  set  of  sensors  missed  13  events  and  correctly  classified  92%  of  the 
data  set.  Other  high  performing  sensor  sets  are  shown  in  Table  3b. 

The  ability  of  the  background  corrected  laboratory  data  to  predict  the  SHADWELL  data 
was  investigated  next.  The  prediction  set,  the  SHADWELL  data,  was  also  background 
subtracted.  In  this  case,  the  earliest  background  times  described  above  were  subtracted  from 
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the  responses  at  the  alarm  time,  and  the  later  background  time.  This  reduced  the  data  set  by 
one  third,  18  fires  and  42  backgrounds/nuisance  sources.  Two  identical  sensor  combinations 
were  tested  with  both  background  correction  and  no  correction.  Oj,  CO50,  ETHYL,  ION  and 
PHOTO  produced  similar  overall  results  for  both  methods;  however,  more  fires  were  missed 
with  the  background  correction.  The  sensor  array  consisting  of  O2,  H2S,  RH,  ION  and 
PHOTO  was  greatly  improved  by  background  correction.  While  there  was  a  big  difference  in 
relative  humidity  during  the  laboratory  and  SHADWELL  tests,  the  change  in  relative  humidity 
was  similar. 

Using  the  VARSELPR,  RH,  ETHYL,  TC,  RION,  and  PHOTO  were  found  to  be  the  best 
set  of  sensors  for  the  SHADWELL  data  set.  The  intra-set  cross  validation  results  for  this  set 
achieved  92%  correct  classification  and  correctly  classified  16  of  the  18  fires  and  39  of  the  42 
background  and  nuisances  sources.  The  misclassified  tests  were  flaming  rags,  welding,  cutting 
steel,  cigarette  smoke,  and  flame  applied  to  pipe  insulation.  These  overall  results  are  excellent 
and  are  similar  to  results  achieved  with  the  laboratory  data  set.  However,  prediction 
performance  when  the  PNN  is  trained  with  the  laboratory  data  is  the  goal  for  this  study. 

Several  sensor  set  combinations  were  investigated  as  shown  in  Table  3c.  The  set  above 
provided  the  best  fire  detection,  but  the  overall  results  were  poor  due  to  the  high  number  of 
false  alarms.  When  the  RION  sensor  was  removed,  the  results  for  both  fire  and  nuisance 
sources  improved.  When  ION  was  substituted  in  the  above  array  for  RION,  the  results 
improved  to  only  12  missed  events;  however,  the  correctly  identified  fires  were  reduced  from 
16  to  12.  Different  combinations  of  these  sensors  were  investigated.  The  best  overall 
performance  using  the  SHADWELL  data  set  as  the  prediction  set  was  achieved  with  the  ION 
sensor  alone.  The  misclassified  fires  were  flaming  rags,  welding,  cutting  steel,  cigarette 
smoke,  flame  applied  to  pipe  insulation,  burning  toast,  and  smoldering  oily  rags  in  a  trash  can. 
None  of  the  sensor  combinations  were  as  strong  as  the  intra-set  results,  suggesting  that  even 
with  background  subtraction  there  are  significant  differences  between  the  laboratory  and 
SHADWELL  data  sets  that  introduce  calibration  transfer  issues.  The  preprocessing  of  the  two 
data  sets  is  different  and  may  have  contributed  to  the  unsatisfactory  prediction  results. 

Further  tests  with  the  processed  data  were  abandoned  due  to  the  uncertainty  of  the  results. 
The  unsatisfactory  results  for  some  of  the  sensor  arrays  were  likely  due  to  the  differences  in 
the  preprocessing  approaches  used  for  the  laboratory  and  SHADWELL  data  sets.  The 
remainder  of  the  report  describes  the  investigations  using  the  raw  data  and  treats  both  the 
laboratory  and  SHADWELL  data  sets  identically. 
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Table  3.  PNN  Classification  Results 


Sensor  Sets 

Number 

Wrong 

Number 
of  Real 
Fires 
Correct 

Number 
of  False 
Alarms 

Overall 

Percent 

Correct 

(a)  0.82%  (No  Background  Correction)  SHADWELL 

(90) 

(18) 

(72) 

CO50ODM  ION 

20 

5 

7 

78 

Temp  RH  ODM  ION  PHOTO 

40 

10 

32 

56 

O2  H2  Ethyl  ION  PHOTO 

45 

11 

38 

50 

O2  CO4000  H2  ION  PHOTO 

46 

11 

40 

50 

O2  CO50  Ethyl  ION  PHOTO 

19 

15 

16 

78 

O2CO50H2S  RH  ION 

72 

18 

72 

20 

O2  H2  Ethyl  RION  PHOTO 

15 

3 

0 

83 

CO  ION  PHOTO 

17 

7 

6 

81 

O2  H2S  RH  ION  PHOTO 

71 

18 

71 

20 

ION  PHOTO 

13 

8 

3 

86 

ODM  ION  PHOTO 

13 

8 

3 

86 

CO50  NO  ION  PHOTO 

18 

4 

4 

80 

(b)  0.82%  (Background  correction)  Laboratory 

(120) 

(82) 

(38) 

CO4000  RH  TC  ODM  PHOTO 

13 

72 

3 

89 

CO50  RH  TC  ODM  PHOTO 

20 

73 

11 

83 

O2  CO4000  RH  ION  PHOTO 

21 

63 

3 

82 

RH,  Ethyl,  TC,  ION,  PHOTO 

20 

69 

7 

83 

O2  H2S  RH  ION  PHOTO 

18 

72 

8 

85 

(c)  0.82%  (Background  correction)  SHADWELL 

(60) 

(18) 

(42) 

RH,  Ethyl,  TC,  RION,  PHOTO  (PNNCV) 

5 

16 

3 

92 

RH,  Ethyl,  TC,  RION,  PHOTO 

24 

16 

22 

60 

RH,  Ethyl,  TC,  PHOTO 

15 

7 

4 

75 

RH,  Ethyl,  TC,  ION,  PHOTO 

12 

12 

6 

80 

TC,  ION,  PHOTO 

7 

15 

4 

88 

ION,  PHOTO 

8 

14 

4 

87 

ION 

7 

15 

4 

88 

RH,  Ethyl 

14 

9 

5 

77 

RH,  Ethyl,  PHOTO 

15 

11 

8 

75 

CO50RH  TC  ODM  PHOTO  ‘B’ 

15 

7 

4 

75 

O2  CO50  Ethyl  ION  PHOTO 

14 

11 

7 

77 

CO50  RH  TC  ODM  PHOTO  ‘A’ 

16 

6 

4 

73 

O2  H2S  RH  ION  PHOTO 

15 

10 

7 

75 

11 


4.2  Raw  Data  Analysis 

The  laboratory  and  SHADWELL  data  sets  were  preprocessed  in  an  identical  manner  for 
this  study  and  all  the  methods  are  applicable  to  real  time  analysis.  Several  studies  were 
conducted  using  the  raw  data.  The  electrochemical  sensors  in  experiment  MV-02  were  prone 
to  large  spikes  due  to  the  Rf  interference  from  the  hand-held  radios  and  interfered  with  the 
studies  conducted  using  the  raw  data.  Rather  than  attempt  to  filter  the  spurious  data,  patterns 
obtained  from  this  experiment  were  removed  from  the  raw  data  analysis.  This  reduced  the 
size  of  the  SHADWELL  data  set  to  87  patterns. 

Intra-Set  Cross-Validation  for  the  SHADWELL  Field  Test  Data 

In  the  previous  study  that  investigated  the  laboratory  data  sets  (reference  (d)),  several  sets 
of  sensors  were  found  to  be  useful  for  accurate  fire  detection  and  nuisance  source  rejection. 
For  these  tests,  sensor  magnitude  and  slopes  were  considered.  In  reference  (d),  the  magnitude 
of  the  sensor  responses  was  sufficient  for  correct  classification.  Similar  experiments  were 
conducted  to  determine  whether  these  same  sets  of  sensors  are  important  for  the  SHADWELL 
field  data.  Training  and  prediction  sets  were  produced  at  the  1 1  and  0.82  %  obs/m  alarm 
levels  as  described  earlier.  An  additional  training  set  was  constructed  at  the  0.82%  level  using 
a  10-point  window  for  the  slope.  For  each  training  set,  VARSELPR  was  performed  to  select 
the  optimal  combination  of  sensors.  Table  4  lists  the  fire  detection  and  nuisance  rejection 
accuracy  of  the  three  best  sensor  combinations.  An  additional  experiment  is  listed  in  which 
only  the  slopes  from  window  methods  4  and  8  (10  point  and  25  point  slopes)  were  available  to 
the  variable  selection  algorithm.  For  comparison,  the  performance  of  the  four  commercial 
Simplex  detectors  is  given  in  Table  5. 

To  further  elucidate  the  differences  between  the  optimal  sensor  combinations  for  the 
SHADWELL  and  laboratory  data  sets,  the  linear  correlation  coefficient  was  computed  between 
each  variable  and  the  correct  classification  of  each  pattern  (1  =  fire,  2  =  not  a  fire).  Table  6 
lists  the  variables  that  had  correlation  coefficients  greater  than  0.5  (absolute  value)  for  each 
training  set. 
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Table  4.  PNNCV  Classification  Performance  for  SHAD  WELL  Field  Test  Data 


Alarm 

Level 

Win 

Size 

Sensors 

Missed 

Events 

Number  Correct 

Fire 

Nuis 

Bkgd 

11% 

8 

APHOTO-B,  ASION-B, 
ASION-A,  AODM-B, 
ARION-B,  PHOTO-A, 
ODM-A 

MV14,  MV15, 
MV20,  MV23, 
MV24 

12/17 

7/7 

63/63 

0.82% 

4 

ASION-B,  PHOTO-A, 

AHjS,  ARH-A,  ACOjo-A, 
ARH-B,  APHOTO-B 

MV6,  MV9, 
MV14,  MV16, 
MV22,  MV25 

16/17 

2/7 

63/63 

0.82% 

8 

ARION-B,  APHOTO-A, 
RION-A,  ARH-A,  ARION- 
A,  O2-A,  ACO50-A 

MV14,  MV15, 
MV  16,  MV20, 
MV22 

14/17 

5/7 

63/63 

0.82 

SLOPES 

ONLY 

4/8 

ASION-B(IO),  ASION- 
B(25),  APHOTO-B(IO), 
AHCI(25),  Aomega(25), 
AH2S(25),  AC05o(25) 

MVl,  MV4, 
MV8,  MV14, 
MV  15,  MV20, 
MV22,  MV23, 
MV27 

9/17 

6/7 

63/63 

Table  5.  Detection  Performance  for  Commercial  Systems  at  the  SHADWELL  Field  Tests 


Sensors 

Setting 

Number  Correct 

Fire 

Nuis 

Bkgd 

PHOTO-A 

11% 

8/17 

5/7 

5/5 

1.63% 

14/17 

2/7 

5/5 

0.82% 

16/17 

1/7 

5/5 

PHOTO-B 

11% 

8/17 

6/7 

5/5 

1.63% 

13/17 

2/7 

5/5 

0.82% 

17/17 

1/7 

5/5 

SION-A 

4.2% 

13/17 

4/7 

5/5 

1.63% 

13/17 

3/7 

5/5 

0.82% 

14/17 

3/7 

5/5 

SION-B 

4.2% 

14/17 

4/7 

5/5 

1.63% 

14/17 

4/7 

5/5 

0.82% 

14/17 

3/7 

5/5 

Table  6.  Ranked  List  of  Linear  Correlation  Coefficients 


Alarm  Level 

Ranked  List  of  Sensors 

11% 

SION-B,  SION-A,  PHOTO-A,  PHOTO-A,  PHOTO-B,  RION-B, 
RION-A,  SION-B,  O2-A,  PHOTO-B,  CO^o-A,  SION-A,  RION-B 

0.82%  -  8 

PHOTO-A,  PHOTO-A,  SION-B,  SION-B,  CO^-B 

0.82%  -4 

(same  as  above  plus  RION-B) 
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Compared  to  the  laboratory  data  in  reference  (d),  the  optimal  sensor  combinations  for 
SHADWELL  are  much  more  heavily  weighted  toward  the  smoke  detectors.  This  might  be  an 
artifact  of  having  a  larger  amount  of  background  data  available  for  training.  The  gas  sensors 
tend  to  have  larger  fluctuations,  which  increase  the  chances  for  false  alarms  on  the 
background.  The  ionization  and  photoelectric  sensors  have  stable  baselines  and  lend 
themselves  to  excellent  background  rejection.  The  results  in  Table  5  clearly  show  that  the 
smoke  detectors  have  poor  nuisance  source  rejection  capabilities.  When  the  sensitivity  setting 
is  low  (11%)  the  Photoelectric  detectors  missed  fires  in  order  to  get  good  nuisance  rejection. 
At  the  most  sensitive  settings,  most  fires  are  detected,  but  at  the  price  of  poor  nuisance 
immunity.  However,  even  at  the  most  sensitive  settings  no  false  alarms  were  detected  in  the 
background  experiments. 

The  list  of  correlation  coefficients  and  the  variables  chosen  by  VARSELPR  suggests  similar 
sensors  to  those  chosen  during  the  laboratory  analysis  (reference  (d)).  Smoke  detectors 
continue  to  be  important  and  the  gas  sensors  that  consistently  find  themselves  being  useful  or 
having  a  high  correlation  with  the  correct  classification  include  CO2,  O2,  CO50,  RH,  and  H2S) 
and  to  a  lesser  extent  (ETHYL  and  HCl). 

Inter-Set  Prediction 

In  these  experiments,  the  laboratory  data  set  was  used  as  the  training  set  and  the 
SHADWELL  data  described  above  were  used  as  the  prediction  set.  The  magnitude  is  an 
average  of  data  points  (5  or  10  total)  and  the  slope  is  calculated  over  10-25  points.  The 
optimal  number  is  unknown,  so  several  different  combinations  were  used  in  these  studies.  The 
various  methods  and  selection  codes  are  given  below: 

Selection  Codes  1  =  magnitude(5) 

2  =  magnitude(lO) 

3  =  magnitude(5)  +  slope(lO) 

4  =  magnitude(lO)  +  slope(lO) 

5  =  magnitude(5)  +  slope(15) 

6  =  magnitude(lO)  +  slope(15) 

7  =  magnitude(5)  +  slope(25) 

8  =  magnitude(lO)  +  stope(25) 

Initial  experiments  involved  sensor  combinations  that  were  useful  for  intra-set  prediction. 
Next,  a  new  program  was  written,  VARSELPR3,  to  perform  VARSELPR  using  an  external 
validation  set  to  compute  the  sum  of  squared  error.  The  prediction  set  (SHADWELL) 
classification  performance  is  used  as  the  criterion  for  selecting  which  sensors  from  the 
laboratory  data  (the  training  set)  are  most  useful.  While  this  method  certainly  biases  the  final 
model  since  the  prediction  data  are  used  for  PNN  optimization,  it  does  provide  a  rough 
estimate  of  what  type  of  classification  is  possible  with  further  data  collection  and  algorithm 
enhancement. 
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Table  7  lists  the  results  from  inter-set  prediction.  The  second  column  lists  the  parameters 
(alarm  level  and  selection  code)  used  for  constructing  the  training  and  prediction  sets.  Column 
3  gives  the  sensors  used.  The  fourth  column  lists  the  PNNCV  results  using  the  training  set 
(laboratory  data).  Columns  5  through  8  lists  the  prediction  results  using  the  laboratory  data  as 
the  training  set  and  the  SHADWELL  data  as  the  prediction  set.  Columns  4  and  5  present  the 
percent  correct  classifications  for  the  intra-set  cross  validation  of  the  laboratory  data  and  the 
prediction  of  the  SHADWELL  data,  respectively.  Columns  6  to  8  show  the  number  of  events 
correctly  classified  per  the  total  number  of  those  events.  The  final  column  gives  a  set  of 
comments  pertaining  to  that  experiment.  In  the  comments  section,  a  description  of  how  that 
set  of  sensors  was  chosen  and  which  board  (A  or  B)  was  used  to  construct  the  SHADWELL 
prediction  set.  The  best  prediction  found  using  an  unbiased  strategy  are  trials  5  and  6,  while 
trial  10  is  the  best  sensor  combination  found  using  a  biased  procedure. 

The  first  two  experiments  used  the  best  sensor  combination  (at  the  0.82  alarm  time  and 
selection  code  #8)  from  our  previous  report.  This  combination  uses  several  sensor 
magnitudes.  As  expected,  this  combination  performs  poorly  in  prediction  due  to  differences  in 
the  background  levels  between  laboratory  and  SHADWELL.  This  is  a  calibration  transfer 
issue.  The  second  set  of  experiments  (rows  3-6  in  Table  7)  takes  optimized  sensor 
combination  from  our  previous  experiments,  but  uses  slopes  rather  than  magnitudes.  This  set 
performs  much  better  and  gives  adequate  performance  compared  to  the  commercial  system  at 
that  sensitivity  setting.  The  next  set  of  experiments  (rows  7-10)  used  the  sensor  combination 
chosen  by  VARSELPRS  using  the  prediction  set  to  guide  the  choice  of  sensors.  The  final  set  of 
experiments  (rows  11  and  12)  used  sensor  combination  chosen  by  VARSELPR3  using  both  the 
prediction  and  CV  error  (sum  of  squared  errors)  as  the  criterion.  This  method  seeks  to  find 
the  sensor  combination  that  performs  well  on  both  the  laboratory  and  SHADWELL  data  sets. 
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Table  7.  Inter-Set  Prediction  Results 


Trial 

Level/ 

Sensors 

%  Correct 

Number  Correct 

Comments 

Win. 

PNNCV 

Prcd. 

Fire 

Nuis 

Bkgd 

Lab 

Shad 

data 

1 

0.82 

COjo,  RH,  PHOTO,  O2, 

92.08 

79.31 

12/17 

2/7 

55/63 

Best  Combo  from 

8 

ACO„  APHOTO 

HAI;  board  A 

2 

0.82 

CO50,  RH,  PHOTO,  O2, 

92.08 

27.6 

17/17 

2/7 

5/63 

Best  Combo  from 

8 

ACO2,  APHOTO 

HAI;  board  B; 
high  false  alarm 
rate  due  to 
differences  in  O2 
magnitude 

3 

0.82 

APHOTO,  ACO2,  ACO50, 

87.08 

86.21 

9/17 

4/7 

62/63 

Slopes  of  best 

8 

AETHYL,  AHCl 

sensors  from  HAI; 
board  A 

4 

0.82 

APHOTO,  ACO2,  ACO50, 

87.08 

85.05 

10/17 

2/7 

62/63 

Slopes  of  best 

8 

AETHYL,  AHCl 

sensors  from  HAI; 
board  B 

5 

0.82 

PHOTO,  ACO2,  AO2, 

90.83 

88.51 

10/17 

4/7 

63/63 

Slopes  of  best 

8 

ACO,o,  ARH,  APHOTO 

sensors  from  HAI: 
board  A 

6 

0.82 

PHOTO,  ACO2,  AO2, 

90.83 

90.80 

13/17 

3/7 

63/63 

Slopes  of  best 

8 

ACOso,  ARH,  APHOTO 

!  sensors  from  HAI: 
board  B 

7 

0.82 

APHOTO,  SION,  AO2, 

85.42 

90.80 

13/17 

3/7 

63/63 

VARSELPR3, 

8 

ASION,  PHOTO,  ARH 

PRED  only;  board 

A 

8 

0.82 

AETHYL,  SION, 

86.67 

93.10 

14/17 

4/7 

63/63 

VARSELPR3, 

8 

PHOTO,  APHOTO,  AO^, 

PRED  only;  board 

AH, 

B 

9 

0.82 

APHOTO,  ETHYL,  CO2, 

82.08 

89.66 

10/17 

5/7 

63/63 

VARSELPR3; 

4 

AHCl,  ANO 

PRED  only;  board 

A 

10 

0.82 

ASION,  APHOTO, 

92.5 

96.55 

14/17 

7/7 

63/63 

VARSELPR3; 

4 

PHOTO,  RION,  CO2, 

PRED  only;  board 

B 

11 

0.82 

APHOTO,  ACO2, 

94.17 

87.36 

8/17 

5/7 

63/63 

VARSELPR3, 

8 

PHOTO,  HCl,  RH, 

Both  PNNCV  and 

RION 

PRED  board  A 

12 

0.82 

26  22  10  44  15  24 

93.33 

87.36 

10/17 

4/7 

63/63 

VARSELPR3, 

8 

ACO50,  PHOTO,  NO, 

Both  PNNCV  and 

APHOTO,  RH,  ACO2 

PRED  board  B 

The  results  shown  in  Table  7  suggest  that  a  combination  of  sensors  can  be  used  to  predict 
both  the  iaiboratory  and  the  SHADWELL  data.  The  combination  shown  in  trials  5  and  6  is 
selected  in  an  unbiased  manner  and  does  well  with  both  the  laboratory  PNNCV  and  the 
SHADWELL  prediction.  The  best  PNNCV  results  obtained  for  the  laboratory  data  (reference 
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(d))  are  around  92%  (at  the  0.82%  level)  and  the  combination  of  PHOTO,  aCOj,  AO2,  ACO50, 
aRH,  aPHOTO  gets  over  90%  of  the  patterns  from  the  laboratory  experiments  correct.  The 
best  prediction  results  come,  as  expected,  from  the  biased  experiments  trials  7-12.  This 
represents  the  best  that  can  possibly  be  done  with  laboratory  data  as  the  training  set.  Table  8 
shows  the  results  using  VARSELPR3  at  the  1.63%  obs/m  alarm  level.  Little  improvement  is 
observed  and  the  results  are  worse  than  the  commercial  systems  at  this  alarm  level.  Ideally, 
the  same  sets  of  sensors  would  be  chosen  with  the  biased  and  unbiased  methods.  Ultimately 
the  best  sensor/algorithm  combination  will  perform  equally  well  on  the  laboratory  and 
SHADWELL  data  sets.  These  results  suggest  that  we  are  close  to  achieving  this  goal. 
However,  the  optimal  sensor  combinations  were  much  different  between  Board  A  and  B, 
which  is  disturbing. 

Table  8.  Inter-Set  Prediction  Results  at  1.63%  Alarm  Level 


Trial 

tf 

Level/ 

Win. 

Sensors 

%  Correct 

Number  Correct 

Comments 

PNNCV 

Lab 

Pred. 

Shad 

Fire 

Nuis 

Bkgd 

1 

1.63 

4 

APHOTO,  NO,  ARION, 
ASION,  CO, 

86.67 

90.80 

11/17 

5/7 

63/63 

VARSELPR3; 

PRED  only; 

Board  B 

2 

1.63 

8 

AETHYL,  PHOTO, 

RION,  AO2,  AOMEG, 
APHOTO 

91.25 

89.66 

13/17 

3/7 

62/63 

VARSLEPR3; 

PRED  only; 

Board  B 

3 

1.63 

4 

ACO50,  PHOTO,  CO,, 
HCL,  RH,  ACO,, 
APHOTO 

94.58 

87.36 

11/17 

3/7 

62/63 

VARSELPR3; 

Both  PNNCV  and 
PRED;  Board  B 

4 

1.63 

8 

ACO50.  PHOTO,  CO,, 
HCL,  RH,  ACO,, 
APHOTO 

94.17 

89.66 

11/17 

4/7 

63/63 

VARSELPR3; 

Both  PNNCV  and 
PRED;  Board  B 

5 

1.63 

4 

ACO5,,,  PHOTO,  CO,, 
APHOTO,  ANO,  HCi 

93.33 

89.66 

1  12/17 

3/7 

63/63 

VARSELPR3; 

Both  PNNCV  and 
PRED*;  Board  B 

6 

1.63 

8 

ACO50,  PHOTO,  CO2, 

HCI,  RH,  APHOTO 

92.92 

86.21 

11/17 

2/7 

63/63 

VARSELPR3; 

Both  PNNCV  and 
PRED*;  Board  B 
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Background  Subtraction  and  Conversion  to  Engineering  Units 


A  common  sensor  preprocessing  scheme  involves  background  subtractions  and  conversion 
to  engineering  units.  In  this  context,  conversion  to  engineering  units  only  pertains  to  the 
smoke  detectors,  as  the  raw  data  for  the  gas  sensors  were  already  in  engineering  units. 
Calibration  models  were  used  to  convert  the  smoke  detector  readings  to  units  of  percent 
obscuration  per  meter.  These  calibration  models  are  based  on  the  calculation  of  I^,  which  is 
essentially  the  average  background  reading.  For  both  the  gas  sensors  and  the  smoke  detectors, 
the  first  30  seconds  of  each  test  was  used  to  determine  the  baseline  reading.  Data  collected  30 
seconds  prior  to  source  initiation  were  used  to  compute  the  background  pattern  for  the 
laboratory  training  set. 

The  same  types  of  experiments  were  performed  on  the  background  subtracted  data  sets  as 
was  done  for  the  raw  data  sets.  VARSELPR3  was  used  to  determine  the  optimal  sensor 
combination  for  various  window  sizes  and  alarm  levels.  The  results  from  these  experiments 
are  shown  in  Table  9.  The  best  results  are  highlighted  in  bold  text  for  each  alarm  level.  The 
classification  results  are  not  much  different  from  those  in  Tables  7  and  8.  In  addition  to  the 
0.82  and  1.63%  obs/m  alarm  levels,  the  11%  alarm  level  was  investigated  and  very  good  PNN 
training  and  prediction  performance  was  observed.  There  is  a  drop-off  in  fire  detection  and 
false  alarm  rate  at  the  lower  alarm  levels. 

The  results  are  somewhat  surprising  since  the  background  subtracted  results  are  no  better 
than  the  results  from  the  raw  data.  Done  properly,  background  subtraction  effectively  solves 
the  sensor  standardization  issue  since  differences  between  the  ambient  background  gas  levels 
between  different  sites  are  subtracted  out.  Any  remaining  deviations  from  baseline  can  be 
attributed  to  the  initiation  of  the  fire  or  nuisance  source  or  just  the  naturally  occurring 
variations  in  the  sensor  output  (either  due  to  sensor  instability  or  changes  in  the  natural 
environment).  Another  interesting  observation  is  that  slope  information  is  less  critical  for 
background  subtracted  data  than  it  is  for  raw  data.  For  raw  data,  the  slopes  were  a  critical 
ingredient  toward  successful  sensor  standardization  (i.e.,  changes  in  sensor  readings  were 
consistent  across  sites  even  if  the  magnitude  was  not)  and  provided  a  convenient  means  of 
using  the  PNN  to  predict  the  SHADWELL  test  data.  In  these  two  studies,  identical  sensor 
combinations  are  not  tested.  However,  the  same  methodology  was  used  in  each  study  to  select 
the  sensor  combinations.  The  differences  in  the  type  of  sensors  selected  suggest  slope  and 
background  subtraction  are  equivalent.  The  next  section  investigates  this  issue  again. 
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Table  9.  Inter-Set  Prediction  Results  from 
Background  Subtraction/Conversion  to  Engineering  Units 


Level 

/Win. 

Sensors 

%Correct 

Number  Correct 

Comments 

PNNCV 

Lab 

Pred. 

Shad 

Fire 

Nuis 

Bkgd 

0.82 

4 

CO,  AETHY,  02, 
PHOTO,  ETHY,  NO 

92.5 

89.65 

13/17 

3/7 

6im 

VARSELPR3, 

Both  PNNCV  & 
PRED  board  A 

0.82 

4 

ODM,  RH,  APHOTO, 
AC02,  H2,  C02 

89.17 

91.95 

10/17 

m 

63/63 

VARSELPR3, 

Both  PNNCV  & 
PRED  board  B 

0.82 

4 

SION,  APHOTO,  C02, 
AOMEG,  02 

86.25 

89.65 

13/17 

H 

63/63 

VARSELPR3; 

PRED  only:  board 

A 

0.82 

4 

SION,  AODM,  C02, 
APHOTO,  ARH 

84.58 

91.95 

12/17 

5/7 

63/63 

VARSELPR3; 

PRED  only;  board 

B 

1.63 

4 

CO,  02,  ACO,  ETHY, 
PHOTO,  NO 

92.5 

89.65 

13/17 

3/7 

62/63 

VARSELPR3, 

Both  PNNCV  & 
PRED  board  A 

1.63 

4 

ODM,  RH,  02,  OMEG, 
AHCL 

93.75 

88.5 

11/17 

3/7 

63/63 

VARSELPR3, 

Both  PNNCV  & 
PRED  board  B 

1.63 

4 

SION,  AODM,  C02, 

NO, APHOTO 

85.41 

90.80 

12/17 

4/7 

63/63 

VARSELPR3; 

PRED  only;  board 

A 

1.63 

4 

SION,  ARION,  C02, 
ASION,  APHOTO,  02 

83.75 

94.25 

13/17 

6/7 

63/63 

VARSELPR3; 

PRED  only;  board 

B 

■ 

CO,  02,  PHOTO, 

SION,  ETHY 

95 

97.7 

15/17 

111 

63/63 

VARSELPR3, 

Both  PNNCV  & 
PRED  board  A 

■ 

ODM,  PHOTO,  02, 
ASION,  IKS 

96.25 

96.55 

13/17 

7/7 

63/63 

VARSELPR3, 

Both  PNNCV  & 
PRED  board  B 

■ 

SION,  AETHY,  CO, 

ACO,  02,  ETHY, 

PHOTO 

95.42 

96.55 

14/17 

in 

63/63 

VARSELPR3; 

PRED  only;  board 

A 

■ 

RION,  SION,  ARION, 

02,  C02,  HCL 

90.83 

96.55 

15/17 

611 

63/63 

VARSELPR3; 

PRED  only;  board 

B 
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Inter-Set  Prediction  with  Real-Time  Playback 

The  experiments  described  above  were  performed  at  discrete  times  for  both  the  training 
and  prediction  data  set.  A  more  relevant  method  of  determining  fire  detection  and  nuisance 
source  rejection  performance  involves  real-time  prediction  of  the  sensor  data.  In  reference  (d), 
real-time  PNN  performance  was  assessed  using  the  laboratory  data  set.  In  these  experiments, 
real-time  PNN  performance  is  investigated  by  training  the  PNN  on  the  laboratory  data  at 
discrete  periods  of  time  and  processing  the  SHADWELL  test  data  as  though  it  were  being 
collected  in  real  time.  These  experiments  will  provide  the  most  robust  test,  to  date,  of  our 
multi-sensor  array  methodology  since  it  demonstrates  two  important  concepts:  (1)  using 
training  data  taken  from  site  and  predicting  data  collected  at  another;  and  (2)  real-time 
analysis.  The  real-time  playback  experiments  will  also  provide  a  means  for  determining 
response  time  to  fires  and  the  ability  to  reject  nuisance  sources  and  sensor  changes  due  to 
changes  in  the  ambient  background  level. 

Thirty-one  different  combinations  of  sensors  were  chosen  for  these  experiments  and  are 
listed  in  Table  10.  The  various  sensor  combinations  were  chosen  based  on  the  results  from 
our  previous  work.  The  majority  of  the  combinations  involved  only  5  sensors,  however,  some 
smaller  sized  arrays  were  also  studied.  In  addition  to  sensor  combination,  several  other 
variables  were  studied.  The  results  shown  in  Tables  7-9  do  not  point  to  an  obvious  conclusion 
regarding  the  use  of  background  subtracted  or  raw,  so  that  option  was  studied  as  well.  For 
earliest  fire  detection  and  highest  nuisance  source  rejection,  the  optimal  training  set  has  not 
been  determined.  To  study  this,  each  sensor  combination  and  preprocessing  scheme  was 
studied  using  PNN  training  sets  constructed  at  the  11%,  1.63%,  and  0.82%  obs./m  alarm 
levels  of  the  Photoelectric  system  from  the  laboratory  data.  A  fourth  training  set  was 
constructed  based  on  time  at  which  five  or  more  sensors  have  deviated  10  times  the  standard 
deviation  of  the  background.  The  final  consideration  was  the  choice  of  using  sensors  from 
board  'A'  or  'B'. 

A  total  of  31  (sensor  combinations)  x  2  (preprocessing  schemes,  raw  or  background)  x  4 
training  set  conditions  (deviation  from  baseline,  11,  1.63,  or  0.82)  x  2  Board  selections  ('A' 
or  'B')  =  496  experiments  were  conducted.  For  each  of  these  experiments,  VARSELPR3  was 
run  to  optimize  the  choices  of  slopes  and  magnitudes.  The  rationale  behind  this  step  is  that 
some  sensors  have  more  informative  slopes  (change  in  sensor  response  as  a  function  of  time) 
than  magnitudes  and  that  the  best  mix  of  slopes  and  magnitudes  is  dependent  upon  the  choice 
of  the  preprocessing  method  and  training  set  composition.  The  best  performing  subset  can  be 
found  in  Table  1 1 .  The  data  were  sorted  according  to  highest  nuisance  source  rejection 
capability. 
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Table  10.  Sensor  Combination  Studied  in  Real-Time  Playback  Experiments 


Combination  It 

Sensors 

1 

PHOTO  SION  CO  0,  ETHY 

2 

PHOTO  SION  CO  CO,  ETHY 

3 

PHOTO  SION  CO,  0,  CO 

4 

PHOTO  SION  CO,  02  ETHY 

5 

PHOTO  SION  RH  OMEG  CO 

6 

PHOTO  SION  RH  OMEG  CO, 

7 

PHOTO  SION  RH  OMEG  0, 

8 

PHOTO  SION  RH  OMEG  ETHY 

9 

PHOTO  RH  CO,  0,  CO 

10 

SION  RH  00,  0,  CO 

11 

PHOTO  ETHY  CO,  0,  CO 

12 

SION  ETHY  CO,  0,  CO 

13 

RION  ETHY  CO,  0,  CO 

14 

RION  OMEG  RH  CO  CO, 

15 

PHOTO  SION  RH  CO  0, 

16 

PHOTO  SION  RION  CO,  CO 

17 

PHOTO  SION  RH  CO,  CO 

18 

PHOTO  RION  RH  CO,  CO 

19 

PHOTO  H,S  CO  CO,  RH 

20 

PHOTO  SION  C02  H,S  RH 

21 

PHOTO  RION  RH  OMEG  ETHY 

22 

PHOTO  ODM  RH  OMEG  CO 

23 

0,  CO  OMEG  RH  SION  PHOTO 

24 

0,  CO  H,  SION  PHOTO 

25 

CO  NO  SION  ODM 

26 

SION  CO 

27 

PHOTO  CO 

28 

PHOTO  SION  CO 

29 

PHOTO  SION  CO  CO, 

30 

SION  CO  CO, 

31 

PHOTO  CO  CO, 
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Table  11.  The  Best  Performing  Sensor  Arrays  for  SHADWELL  Prediction 


Combination 

ft 

Alarm  Time 
1=0.82% 
2=1.63% 

3  =  11% 

4= temporal 

Prepro 

cessing 

1  =  bkg 

d 

2  =  raw 

Board 

A/B 

1=A 

2=B 

Total 

Number 

of 

BKGD 

Tests 

Correct 

(29) 

Total  Number 
of 

Fire/Nuisance 

Tests 

Correct 

(29) 

#of 

Fires 

Correct 

(17) 

#of 

Nuisance 

Sources 

Correct 

(7) 

#of 

BKGD 

Tests 

Correct 

(5) 

20 

3 

2 

2 

29 

24 

15 

4 

5 

19 

2 

2 

1 

29 

23 

14 

4 

5 

17 

3 

2 

1 

29 

22 

13 

4 

5 

25 

2 

1 

2 

28 

23 

14 

4 

5 

30 

1 

1 

1 

28 

23 

14 

4 

5 

19 

2 

2 

2 

27 

23 

14 

4 

5 

20 

1 

2 

1 

26 

24 

15 

4 

5 

5 

3 

2 

2 

29 

24 

16 

3 

5 

20 

2 

2 

1 

29 

24 

16 

3 

5 

6 

3 

2 

1 

29 

23 

15 

3 

5 

6 

3 

2 

2 

29 

23 

15 

3 

5 

16 

3 

2 

2 

29 

23 

15 

3 

5 

17 

3 

2 

2 

29 

23 

15 

3 

5 

19 

1 

2 

2 

29 

23 

15 

3 

5 

19 

3 

1 

2 

29 

23 

15 

3 

5 

19 

3 

2 

1 

29 

23 

15 

3 

5 

20 

3 

2 

1 

29 

23 

15 

3 

5 

26 

3 

1 

1 

29 

23 

15 

3 

5 

26 

3 

2 

1 

29 

23 

15 

3 

5 

27 

3 

2 

1 

29 

23 

15 

3 

5 

28 

1 

2 

1 

29 

23 

15 

3 

5 

28 

3 

1 

1 

29 

23 

15 

3 

5 

28 

3 

1 

2 

29 

23 

15 

3 

5 

29 

3 

2 

2 

29 

23 

15 

3 

5 

25 

3 

1 

2 

29 

22 

14 

3 

5 

26 

3 

1 

2 

29 

22 

14 

3 

5 

26 

3 

2 

2 

29 

22 

14 

3 

5 

28 

3 

2 

1 

29 

22 

14 

3 

5 

28 

3 

2 

2 

29 

22 

14 

3 

5 

30 

3 

2 

2 

29 

22 

14 

3 

5 

The  best  performing  sensor  combinations  are  highlighted  in  bold  text  above  and  listed 
here. 

5  PHOTO  SION  RH  OMEG  CO 
17  PHOTO  SION  RH  CO2  CO 

19  PHOTO  H2S  CO  CO2  RH 

20  PHOTO  SION  C02  H^S  RH 
25  CONOSIONODM 

30  SIONCOCO2 
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Sensor  combinations  19  and  20  without  background  correction  appear  more  than  once  with 
different  board  and  alarm  time  selections.  The  deviation  from  baseline  or  temporal  method  to 
determine  the  training  pattern  was  not  present  in  any  of  the  top  performing  combinations.  All 
of  the  combinations  include  a  smoke  detector  and  sensors  that  have  been  identified  in  earlier 
studies  as  top  performers.  The  overall  correct  classification  results  are  similar  to  the  intra-set 
PNN-CV  described  with  both  raw  and  processed  approaches  with  19  out  of  24  fire/nuisance 
sources  correctly  processed.  Based  on  the  results  above,  combinations  19  and  20  would  be 
selected.  However,  both  of  these  combinations  use  the  hydrogen  sulfide  sensor.  The  long 
term  stability  of  this  sensor  is  a  concern  because  the  calibration  gas  is  unstable  and  expensive. 
Therefore,  combinations  5  and  17  were  selected  due  to  the  robust  sensors  in  the  arrays.  The 
misclassified  events  for  each  of  these  arrays  are  listed  in  Table  12. 

Table  12.  Misclassifled  Events  for  Each  Candidate  Suite 


PHOTO  SION  RH  CO^  CO 

PHOTO  SION  RH  OMEG  CO 

Burning  Toast 

Cutting  Steel 

Cigarette  Smoking 

Smoldering  cotton  sheet,  pillow,  wool  blanket,  and  mattress 
Smoldering  pillow  in  a  pillow  case 

TODCO  Wallboard  exposed  to  a  flame 

Pipe  insulation  exposed  to  a  flame 

Burning  Toast 

Cutting  Steel 

Cigarette  Smoking 

Welding 

Smoldering  pillow  in  a  pillow  case 

5.  CONCLUSIONS 

This  report  describes  an  important  step  towards  designing  an  early  warning  fire  detection 
system.  It  investigates  the  ability  of  an  algorithm  trained  on  laboratory  data  to  predict  events 
at  another  location  with  different  environmental  parameters.  Sensor  combinations  were 
identified  that  provide  similar  overall  classification  results  between  the  intra-set  studies  and  the 
inter-set  study.  Using  the  extensive  playback  investigation  examining  more  than  400  variables, 
two  candidate  suites  were  selected  for  prototype  development: 

PHOTO  SION  RH  OMEG  CO 

PHOTO  SION  RH  CO^  CO 

The  decision  was  difficult  because  there  were  several  very  good  different  combinations. 
Therefore,  five  additional  sensors  were  selected  to  include  in  the  fiiture  test,  NO,  HjS, 
ETHYL,  RION,  and  Oj.  This  will  allow  maximum  flexibility  in  the  future  algorithm 
development.  Much  work  remains  in  optimizing  the  algorithms  for  early  fire  detection  for  the 
best  performance,  however,  the  results  continue  to  be  encouraging. 


23 


The  overall  classifieation  results  of  this  study  are  not  as  good  as  in  the  previous  work. 
While  the  multi-criteria  detection  system  consistently  performs  better  than  the  commercial 
photoelectric  detector,  the  performance  is  similar  to  the  ionization  detector.  The  nuisance 
source  rejection  in  this  study  is  nearly  identical.  Due  to  the  limited  data  set  and  lack  of 
replicates  it  is  difficult  to  access  why  this  is  the  case.  As  time  goes  on  and  more  shipboard 
fires/nuisance  conditions  are  measured,  the  overall  performance  is  expected  to  result  in  better 
predictions.  However,  observations  made  of  the  fire/nuisance  tests  in  progress  made  it  clear 
that  the  definition  of  nuisance  source  is  a  difficult  decision.  The  burning  toast  experiment  was 
extremely  smoky  and  a  small  fire  was  produced  in  the  toaster.  Inclusion  of  items  like  this  test 
in  the  training  set  will  influence  the  predictions.  If  one  looks  at  the  results  from  reference  (c) 
presented  here  in  Tables  13  and  14,  similar  nuisance  sources  are  missed.  However,  the  earlier 
study  had  multiple  replicates  of  each  test  and  for  all  but  cutting  steel  and  grinding,  only  one 
replicate  test  was  missed  by  the  multi-criteria  detection  system. 

In  addition,  there  are  differences  in  the  experimental  design  of  the  laboratory  and 
SH  A  DWELL  tests  that  generated  the  two  data  sets  used  here.  The  change  in  experimental 
design  demands  a  new  measure  of  success,  and  that  should  be  time  to  alarm.  In  the 
fire/nuisance  source  tests  completed  in  the  laboratory,  many  of  the  small,  incipient  fires  were 
not  detected  by  the  commercial  fire  detection  systems.  These  systems  are  typically  set  at  an 
alarm  level  (11-8%  obs/m)  because  it  provides  the  best  nuisance  source  rejection  and  will 
detect  a  fire  when  it  exceeds  a  given  threshold.  In  the  SHADWELL  tests,  the  fires  were 
larger  in  nature  and  were  sustained  long  enough  to  produce  an  alarm  by  the  commercial  fire 
detection  systems  for  most  of  the  tests.  Therefore,  a  direct  comparison  of  correct  classification 
as  a  measure  of  success  is  less  useful  here.  The  time  to  alarm  would  give  more  information 
about  system  performance  and  is  more  closely  related  to  the  goals  of  the  program  to  produce 
an  early  warning  fire  detection  system.  The  differences  in  these  two  data  sets  need  careful 
examination  and  will  be  the  subject  of  future  tests  and  reports.  As  algorithm  development 
proceeds,  alternative  measures  of  success  will  be  used  in  optimizing  the  methods. 
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Table  13.  Misclassified  Events  Using  CO4000J  MICX,  ODM  and  RION 
at  the  Photoelectric  0.82%  Obs./m  Alarm  Level 


Test  ID 

Scenario  Type 
(Real/Nuisance) 

Source  Description 

DCAS029 

Real 

Propane  Bunsen  burner 

DCAS055* 

Real 

Smoldering  Pillow,  with  pillow  case 

DCAS043 

Real 

TODCO  wall  panel 

DCAS077 

Nuisance 

Burning  Toast,  one  slice 

DCAS083 

Nuisance 

Grinding  cinder  block 

DCAS084 

Nuisance 

Grinding  cinder  block 

DCAS085 

Nuisance 

Cutting  steel  with  acetylene  torch 

DCAS087 

Nuisance 

Cutting  steel  with  acetylene  torch 

DCAS088 

Nuisance 

Cutting  steel  with  acetylene  torch 

DCASlOl 

Real 

Smoldering  electrical  cable  -  LSTHOF-9 

DCAS106 

Real 

Smoldering  electrical  cable  -  LSTPNW- 1-1/2 

DCAS107 

Real 

Smoldering  electrical  cable  -  LSTPNW- 1-1/2 

DCAS116 

Real 

Propane  Meker  burner 

DCAS0132 

Nuisance 

Smoking  12  Cigarettes 

*  This  test  is  one  of  four  similar  tests.  Examination  of  the  sensor  responses  indicates  very  small 
responses  and  a  response  pattern  unlike  the  other  replicates.  In  fact,  this  test  was  not  a  replicate 
at  all  because  the  heating  rod  was  placed  under  the  pillow  rather  than  on  top  as  with  the  other 
tests. 
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Table  14.  Misclassified  Events  Using  Oj,  H^S,  RH,  ION  and  PHOTO 
at  the  Photoelectric  1.63%  Obs./m  Alarm  Level 


Test  ID 

Scenario  Type 
(Real/Nuisance) 

Source  Description 

DCAS029 

Real 

Propane  Bunsen  Burner 

DCAS055* 

Real 

Smoldering  Pillow 

DCAS074 

Nuisance 

Grinding  Steel 

DCAS088 

Nuisance 

Cutting  steel  with  acetylene  torch 

DCASllO 

Real 

Igniting  electrical  cable  with  a  torch  -  LSDSGU-14 

DCAS116 

Real 

Propane  Meker  burner 

*This  test  is  one  of  four  similar.  Examination  of  the  sensor  responses  indicates  very  small 
responses  and  a  response  pattern  unlike  the  other  replicates.  In  fact,  this  test  was  not  a  replicate 
at  all  because  the  heating  rod  was  placed  under  the  pillow  rather  than  on  top  as  with  the  other 
tests. 
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